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Abstract 

In this paper, a result of Albert, Atkinson, Handley, Holton, and Stromquist (Propo- 
sition 2.4 of PP) which characterizes the optimal packing behavior of the pattern 1243 is 
generalized in two directions. The packing densities of layered patterns of type (l a , a) and 
(1, 1,(3) are computed. 

AMS Subject Classifications: 05A15, 05A16 
Keywords: pattern containment, permutations, layered permutations, packing density. 



1 Definitions and Notation 

Let cr be a permutation on [l..n], and let r be a pattern on [l..m], for some m < n. We say that 
an m-subset S C [l..n] is an occurrence of r in a if the restriction of a to S is isomorphic to r, 
that is, they are in the same linear order. There have been two major areas of study within the 
realm of pattern containment, avoidance and packing. The usual aim of the pattern avoidance 
problem is to enumerate or otherwise characterize the permutations on [l..n] which avoid a 
specific pattern or set of patterns. In contrast, the study of packing patterns into permutations, 
which was begun in the early 1990s as an offshoot of the study of pattern avoidance, aims at the 
opposite question: Given a fixed pattern r (or set of patterns, as in pP), how must we structure 
a permutation a of length n so that it has the maximal number of occurrences of r? Much of 
the founding work on the subject was done by Alkes Price, in his Ph.D. dissertation (0), and 
most of the literature on the subject addresses the class of layered permutations, which we will 
define below. As jH| is not widely available, the reader is referred to [2j, which addresses a broad 
range of topics related to pattern containment. 





Figure 1: The permutation 321549876 is layered, with layer sizes (3,2,4) 
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Figure 2: (a) Notice that in each occurrence of the pattern r = 32154, each layer of r must 
be contained in a layer of the larger permutation [5,4,3,2, 1,7,6, 10,9,8]. An occurrence of r 
is circled, (b) Observe how the antilayer in the pattern r = 12543 may 'climb' layers in the 
larger permutation [2, 1, 4, 3, 6, 5, 10, 9, 8, 7] rather than being contained in a single antilayer. An 
occurrence of r is circled. 



Adopting the notation of Price, we define a layer in a to be a contiguous decreasing subse- 
quence of consecutive integers. A pattern is called layered if it consists of an increasing sequence 
of (disjoint) layers (a layered permutation is shown in figure [Q. 

It has long been thought that the optimal packing behavior of patterns containing a con- 
tiguous block of r layers of size 1 was analagous to that of patterns having one layer of size r in 
the corresponding place. In P, the authors were the first to address nontrivial patterns having 
multiple layers of size 1 in a row, with the example of the pattern 1243. The authors of that 
paper showed that the optimal packing behavior of 1243 is analogous (in its simplicity) to that 
of 2143, that is, there is a single increasing block, followed by a single decreasing block. In this 
paper, we provide two slight generalizations of their work, to patterns having layer sizes (l a , a) 
and patterns having layer sizes (1, l,/3), both of which display similar behavior. 

To deal efficiently with contiguous blocks of layers of size 1, we define an antilayer to be 
a contiguous increasing subsequence of consecutive integers, that is, a contiguous sequence of 
layers of size 1. The more general term block will be understood to mean either a layer or an 
antilayer. We will use the term isolated point to describe a layer of size 1 between two layers 
of larger sizes. Since [3] settles the question of optimal packing behavior for patterns having 2 
layers, the next logical step is to attack the slightly more general class of patterns having two 
blocks. The only such patterns not characterized by Price's work are those which consist of one 
antilayer and one layer [each of size at least 2] . 

While the concepts of a layer and an antilayer look virtually the same, they unfortunately 
must be handled in quite different ways, given the traditional approach to the problem. Due 
to the work of Price and Stromquist, when r is a layered pattern, we are able to restrict our 
search for r-maximal permutations of [l..n] to the much smaller class of layered permutations 
of [l..n]. Note, however, that in an occurrence of the pattern r in a permutation a, any layer of 
r must be contained in a layer of a, but antilayers in r need not be contained in antilayers of 
a, as they can 'climb' a list of several layers, having one element in each (see Figure EJ). This 
simple problem is enough to make the computation notably more difficult, so that a slightly 
more delicate argument is required to prove a conjecture which at the outset seems obvious. 
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For convenience of notation, let r a ^ be the layered pattern having layers (l a ,f3). We will 
adopt some of Price's notation for the counting of occurrences: for a G S n and any r G S m , 
m < n, let g(r, a) denote the number of occurrences of r in a. Let 

g(r,n) = max^r, a), 

(T&Sn 

and say that a G S n is r-maximal if g(r, a) = g(r, n). 

A theorem attributed to Galvin (reproduced as Theorem 2.1 of pj) states that the sequence 

^fey- ) is in fact decreasing in n, so that the limit 

w J n>m 

exists. This limit, which we will denote p(r), is called the packing density of r. We will 
compute the packing density of each class of patterns we explore. 



2 The Layered Pattern r a , a 



The idea in this section is that given the packing behavior of a small pattern, we may determine 
the packing densities for larger, similar patterns by exploiting the similarity in structure. 

2.1 Proposition: The structure of the maximizing permutation of size 2n for the pattern 

T a ,a '■= [1, 2, ... a, 2a, 2a — 1, ... a + 1] 

is invariant of a, that is, the r^Q-maximizing pattern of length 2n is of the form [1, 2, ... n, 2n, 2n— 
1, . . . , n + 1] for all a. 

Proof. The case a = 2 was proven in Proposition 2.4 of it follows that the maximal number 
of 1243s in a pattern of length 2n is (™) . In each step, we will show that if the pattern 
[1,2, ... ,n, 2n, 2n — 1, . . . , n + 1] is not r aja -maximal, then in fact it cannot be T2 i 2-uiaximal, 
which would contradict the known result for 1243. We will first prove the a = 3 case. Consider 
T3 t 3 = 123654, and suppose that g{rz$, 2n) > Q) + 1. Let a G be rs^-maximal. In each r^s, 

there are 9 = 2 instances of 1243s. Suppose that a is an instance of 1243 in a. Now, each of 
the remaining 2n — 4 elements, if in a containing a, can be in either side, the 123, or the 
654, but not both, so there are i ■ (2n — 4 — £) ways to form a r 3j3 for some i. Since the largest 
value of the expression i ■ (2n — 4 — £) occurs when £ = n — 2, it follows that the number of r 3 3 s 



containing a 1243 is at most 
it must have at least 



n 



2) 2 . Hence, if a has at least Q)' 

© 2 



1 occurrences of r 33 , then 



(n - 2)' 



+ 1 



> 



occurrences of 1243, contradicting pQ. 

The general case is quite similar: Suppose that g{r a 



2n) > (?) 2 + 1 and let a G Sm be a 



permutation having g{r a ^ a ,2n) occurrences of T a ^ a . We will again count the number of 1243s in 
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a: In each occurrence of r a>a , there are (") occurrences of 1243. Given a particular occurrence 
To of 1243, we need to determine the maximum number of r ajQ s which could contain To; however, 
for each r Q a containing To, the other 2a — 4 of its elements must be a r a _2, Q _2 in the remaining 
2n — 4 elements of a. That is, the number of occurrences of r Q Cf which can contain To is bounded 

above by g(T a _2,a-2i 2n — A). By induction, there are at most ( n In) °f these. Hence, the number 
of 1243s in a is at least 



/n-2V 
\a-2J 



+ 1 



where e > 0, which again contradicts the result of p. It follows that g{r a)Cn 2n) < (") for each 
n, which means that the structure of a r^a-maximizing permutation on [1..2n] is of the form 
[l,...,ra, 2n, 2n- l,...n+ 1]. ♦ 

2.2 Corollary: The packing density of the pattern t Q)CC above is 



P(Ta, 



)2q 



Proof. The result follows from taking the limit in n of the fraction 



g(T a ^2n) 



C 



)C) 



These results are nice enough, but they still only apply to a very narrow class of patterns. 
The next step forward from here is to try to prove a statement about patterns of the same 
format, but with 2 blocks being permitted to have different sizes. 



3 The Layered Pattern T2 t p 

As the optimal packing behavior of the layered pattern r a ^ does not in general adhere to the 
same degree of symmetry as the case a = 0, the ideas of the previous section do not so easily 
lend themselves to the general case, so for the duration of this section we must cease to rely on 
known results and do some computation of our own. 
We will later need the following technical lemma: 

3.1 LEMMA: Let k, £, m, n G N s.t. k < £ < m < n. Then, we have 

r)(7M;)Cr 

That is, we can have more combinations if bigger sets choose the bigger subsets. 

Proof. Although I am sure there is an elegant combinatorial proof, the following straight com- 
putation will do, as this result is only developed here as a tool for later use. We have 

( m ) k\ 
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and 

■^ = ~(n-l)(.. .)("-' + !) 

so that the result follows from cross-multiplying. ♦ 

For the remainder of this section, let f3 > 3 and let a G S n be a permutation which is 
T2 ij g-maximal. 

An early result due to Stromquist provides us with the first important assumption we can 
make about a, namely that it is layered. 

3.2 Theorem (Theorem 2.2 of [5J): Let rGS m bea layered pattern. Then, for each n> m, 
we have 

g(r, n) = m&x{g(r, a) :: a G S n } = max{g(r, a) :: a G S n is layered} . 



3.3 Lemma: We can assume without loss of generality that a has no isolated points, i.e. layers 
of size 1 between two layers of size > 2. 

Proof. Suppose we have an isolated point in between layers Li and L i+ i. Switching the positions 
of Li and the isolated point does not decrease the number of occurrences of r, so we may as well 
do that. Since we can always move an isolated point to the left of a layer, eventually all isolated 
points will be soaked up by antilayers. We can assume, then, that a has no isolated points. ♦ 

It follows now that a is a sequence of layers and antilayers, with no isolated points. In fact, 
a more general statement is true by the same logic: 

3.4 Lemma: We may assume that a consists of a single antilayer A\ followed by a list of 
(nontrivial) layers L l5 . . . , as shown in Figure El 

Proof. Suppose that in the middle of the permutation a we have an antilayer between two layers. 
Similar to the proof of Lemma 13.31 if we switch the positions of the antilayer and the layer to 
its left (see figure 0]), all the occurrences of r 2j/ g that were originally there are left intact, and we 





















X 















Figure 3: The form of a, as shown in Lemma [3.41 
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Figure 4: There can be no occurrences of t<i$ inside the boxed area, so it will not destroy any 
occurrences of T2,p to switch the positions of L a and A. 



create more as long as the antilayer has length at least 2 and the left layer has length at least 
(3. Thus, in a permutation a, we may move all antilayers to the left of all layers without losing 
any occurrences of T2,/?, achieving the desired layer/antilayer pattern. ♦ 

3.5 Lemma: The layers Li, . . . , may be assumed to be in nondecreasing order by size. 

Proof. Suppose L; and L i+ i are adjacent layers, and that Lj is larger than L i+ i. What happens 
if we switch the positions of the two layers? Unless an occurrence of has its /3-layer in one 
of Li or L i+ i, it will be preserved, just moved, so we need only worry about those occurrences of 
T2 } p whose /3-layer is contained in either Lj or L i+ \. Let y — \Ai\ + X^=i l-M be the number of 
elments of a to the left of L i: and let a be the number of increasing 2-sequences in this range. 
Then, the number of occurrences of r 2j /3 which have a /3-layer in Lj or L i+ i is 

f\Li\\ f\L i+ i\\ f\L i+ i 



J V P J \ P 

If we swap the layer-lengths, the number of these occurrences becomes 



J V J V 

Of course the sum of the first two terms remains the same. However, since Lj > L i+1 , the final 
term is certainly larger after the switch, by our technical lemma. It follows that if |Lj| > 
the permutation a cannot be T2 5j a-maximal. Since this statement holds for all i 6 [l..k — 1], we 
can assume that if a is maximal, then its layer-lengths are in increasing order. ♦ 

3.6 Lemma: If \o~\ > 2 + (3, then we can assume that a begins with an antilayer of size at least 
2 and ends with a layer of size at least (3. 

Proof. By Lemmas 13.41 and 13.51 if is not at least as big as (3, or if A\ is not at least as big as 
2, then there can be no occurrences of in a. Assuming a is T2 >/ 3-maximal, this cannot be the 
case. ♦ 

3.7 Lemma: We may assume that \Lk\ > 
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Proof. Suppose that \A\\ > \Lk\, and suppose we remove the last element x of A% and place 
it at the beginning of L^. We will show that there is a (strict) net increase in the number of 
occurrences of T2,p, so that a cannot have been T2 i/ 3-maximal. 

First, we count the occurrences which are lost when we remove x: 

(1) if x is a first element, then the second element is in some Lj and the layer is in some Lj, 
j > i: the number of these is 



l<i<j<k 



(2) if x is a second element, then the first element must have been an earlier element of A\, 
and the layer could be in any Lf. these are enumerated by 



1=1 

Of course x is never in a /3-layer because x G A\. 

Next, we count the occurrences which are gained by putting x into L^. In all new occurrences, 
x must be in a new /3-layer, the other elements of which can be any (3—1 elements from L^. 
The first two elements must come from one of three places: 

(1) both from A\. there are 

\M\-l\(\L h \ 

2 J\J3-: 

of these. 

(2) one from Ai, and one from another layer: there are 



& 

of these. 

(3) from two different layers and Lf there are 



i=i 







l<i<j<k-l 



J I 



of these. 
Hence the loss is 



1=1 v ^ 7 l<i<j<k v ^ 
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and the gain is 



(1 ) ™-w(£[yrrm 



E 

l<i<j<Jfe— 1 



Li La 



\Lk\ 



Naturally, to provide a contradiction to T2 i/ g-maximality of a, we will be showing that the gain 
must exceed the loss. It will later expedite our computation to consider separately the cases of 
occurrences of r 2j/ g which have a /3-layer in L k : we can write the loss as the 4-term sum 



fc-i 



(2) 



i=l 



Li 

P 



P 



E 

l<i<j<k-l 



LA 



P 



k-l 

Eft 



i=l 



Lk 

P 



Now, notice that for each j e [l..k] we have 



(3) 



P 



L, 



P \0 



< 



Lh\ ( \Lh\ 



P \P-\ 



because of our assumption that the layers are ordered by increasing size. Now, we have 

k—1 

(MM-i)5>* 



i 

P 



i=i 



and 



fc-i 

E^. 



i=l 



L k 

P 



k-l 

<Eii 

i=i 
1 

~P 
1 

< - 

" P 



\L k \ 



P \P-l 

k-l 

Lk I | L 



P 



(by 



i=l 



\Lk\ 
P-l 



k ~ l / I r I 

i=l 



because we are assuming also that \L k \ < \A]\. Hence, we are able to bound the sum of the first 
and last terms of (0) strictly below the first term of (JTJ because of our assumption that P > 3. 
That the third term of (J2J) is bounded below the third term of JH) is clear from our relation (JHJ) 
on binomial coefficients. Finally, we may bound the second term of (0) by raw computation: By 
Lemma 1331 we may assume that \L k \ > P > 3, so that \Ai\ > 4 and thus \Ai\ — 1 < § (\Ai\ — 2). 
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It follows now that 

(Ki-i) c^ 1 ) <(^ii-i) 

<(IA|-i) 
<(l^i|-i) 
<(l^i|-i) 

We have thus bounded the loss strictly below the gain, so the change must have resulted in an 
increase in the number of occurrences of r. In particular, to assume that > \L k \ would be 
in contradiction to the T2„«-maximality of a. We may assume, then, that \L k \ is at least as big 
as \A\\. ♦ 

Finally, we may begin piecing together the information we have gathered about a to gain an 
important result: 

3.8 Theorem: For r e S m , let 

(4) gk{T,n) — max {<7(t, a) :: a G S n is of the form A\L\ ...L k and \L k \ > 

that is, maximize only over the permutations a having k nontrivial layers which satisfy Lemma 
15.71 Then, for each k > 2, we have 

9kfo,i3,n) < gi(r 2 ,p,n). 

Proof. It will suffice to show that g k {ji,pi n) < gk-i{ T 2,p> n) for k > 2. Let a = A\L\ . . . L k e S n 
be a permutation for which the expression (j3J) is maximized, and assume k > 2. In this context, 
we will not need the full strength of Lemma f3. 41 simply write a = AiLia*, as in Figure El We 







a* 




\ 











Figure 5: Less-specific structure of the permutation a 



\Lk\ ( l-^fcl \ 

T\p-y 

\Lk\ I \Lk\ \ 
~3~\J3-lJ 
(Kl - 1) / \L k \ \ 

3 \P-l) 
{\A l \-2) (\L k \\ 
2 \f3-l) 
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will show that replacing A\L\ with a single antilayer of size + \L\\ does not decrease the 
number of occurrences of r. 

Let A denote the number of occurrences of Ti$ that we lose when we remove A\ and L±, and 
let r be the number of occurrences we gain when we add in the antilayer. Now, an occurrence 
of T2 /3 is lost whenever it has at least one element in A\ U L\. In this case, either it can have 
its first two elements in A\ and a layer of size (3 in Li, its first two elements in A\ and a layer 
of size (3 in cr*, its first two elements from A\ and Lx, respectively, and a layer of size (3 in cr*, 
or a single element in A\ U Lx and the rest of the occurrence (i.e. an occurrence of T\^) in cr*. 
Accordingly, we have 



A 



At 
2 



9(to. 



a 



At 
2 



l^iPil 



+ g(T 1J3 ,<r*) [\At\ + \Lt\ 



When we place an antilayer of size \A\\ + \L\\ at the beginning of a*, we create two kinds of 
occurrences. First, any two elements in the new antilayer, together with any layer of size (3 in 
cr*, will create an occurrence of r 2j/ 3. Also, any element of the new antilayer in concert with an 
occurrence of in cr* will create a new occurrence of Ti$- It follows that 



r = g(T 0>p ,a*) 



Ail + \Lt 



+ g(T 1> p,a*)[\A 1 \ + \L 1 \] 



so that the net gain is 



(5) 



r-A 



SV0./9, a 



A x \ + \L X 
2 



2 



l^ill^il 



2 



Lx 

(3 



(the terms involving g{ri s p, a*) cancel out). Now, in the first term of (0), the expression ^l j4l l+l Ll ^ 
counts the number of ways to choose 2 elements from Ax U Lx, and we subtract off the number 
|Al||Li| of ways of choosing one from each set and also the number ( g ) of ways to choose both 
from Ax- If we don't choose both elements from A\ and we don't choose one from each set, we 
must choose 2 elements from \Lx\, so the first term of (0) becomes 



Lx 
2 



Hence, we have 
(6) 



r-A = ^(r , /3 ,a*)( |L 2 l1 ) 



2jU 



We must show, then, that the expression © is always nonnegative whenever a* is nonempty. 
Suppose first that \L X \ > \A X \. Then, we have 



r-A 



> 



Lx 
2 

Ax 
2 

Ax 
2 



g(T ,(3,a*) - 



2 ){(3 

Lx 

P 
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Since ( 2 ) i s always positive, it is sufficient to prove that g(r 0)l 3,a*) > ); however, this 
inequality holds whenever a* has at least one layer as large as L\, which follows from Lemma 
1531 

In the case that \L X \ < we know that ( A j) ( |L ^) < ( |L 2 l1 ) (^) by our technical Lemma 
()3.1j) . so we have 



r- a 



> 



Li 

2 

Li 
2 

Li 
2 



#Oo,/3,^*) 

^( r 0,/3,^*) 

#(jo,/3, a* 



2 
2 



(3 

M 



A, 

(3 



which is nonnegative whenever a* has a layer at least as large as |^4i|. However, this fact is 
guaranteed by our adherance to permutations satisfying Lemma 13.71 

We have thus reduced the number of nontrivial layers by 1 without decreasing the number 
of occurrences of r 2i/ 3, so that if a T 2j/ g- maximal permutation a has k nontrivial layers, it still can 
have no more occurrences of t 2j/ 3 than a r 2ij g-maximal permutation having k — 1 nontrivial layers; 
the result follows. ♦ 

The main results of this section now follow as corollaries: 

3.9 COROLLARY: There is a pattern a G S n which maximizes the number of occurrences of r 2>/ g 
and which consists of a single antilayer followed by a single layer. Hence, the maximum number 
of occurrences of T2./3 in a permutation in S n is 



(7) 



g(r, n) = max 

x„€[0..n] 



n — x r 

(3 



Proof. By Lemmas 13 1 41 and 13 . 71 we have 

g(i~ 2 ,p,n) = m&xg k (T 2tf 3,n). 

k<n 

From here, it follows from Theorem 13.81 that g , (r 2ij a,n) = gi(T 2 ^,n), which is clearly (J7j). ♦ 
3.10 Corollary: The packing density of r 2ij g is 



(3 + 2 
2 



P + 2J \(3 + 2 







Proof. Notice that we could easily rewrite the expression ((7J) as 



(8) 



9\t, n ) 



max 

£nG[0,l] 



\Hnn\\f [(I -Qn} 

(3 



The work of Price ([5], Theorem 3.1) shows that a sequence (£ n ) which maximizes the expression 
(JHJ) approaches a constant £, that is, there is an asymptotically best ratio of the sizes of the layer 
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and the antilayer of a. For large enough n, we approximate £ n by the constant value £. It follows 
that 

p{T 2 ,p) = hm - 
= lim 

n 

(9) 



(^2 ^ ( L(1_€)nJ ) 
(3+2) 



(gn) 2 ((l-g)n)' 3 

lim - 2 ' 



09+2)1 



/5 + 2 
2 



We may now maximize the expression (0) via elementary calculus. Noting first that © evaluates 
to when £ is or 1, we may maximize the expression over £ e [0, 1] by simply setting its 
derivative (with respect to £) to 0: setting 

^ [£ 2 (i - 01 = £ [2(1 - - #(1 - OH = 0, 

we have 

2(1 - = # 

since the solutions £ = and £ = 1 are clearly unfavorable, so that £ = The result 

follows. ♦ 
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